{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "BP神经网络也叫做误差反向传递网络\n",
    "\n",
    "BP神经网络基本原理图：\n",
    "![image-2.png](https://img-blog.csdn.net/20180720162033746?watermark/2/text/aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RhYWlrdWFpY2h1YW4=/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "BP神经网络分为输入层，隐含层，以及输出层。如上图：\n",
    "以下i , j ,k均代表下标\n",
    "\n",
    "对于输出层节点i：\n",
    "                                        \n",
    "                                            δi = yi(1-yi)(ti-yi)\n",
    "\n",
    "对于隐藏层节点δi:\n",
    "                                         \n",
    "                                            δi = ai(1-ai)∑Wki*δk\n",
    "\n",
    "\n",
    "最后，更新每个连接上的权值：\n",
    "\n",
    "                                            \n",
    "                                            Wji ⬅ Wji + ηδj * Xji\n",
    "其中η为学习率（学习率过大和过小都不适合拟合）"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们取网络所有输出层节点的误差平方和作为目标函数：\n",
    "                                            \n",
    "                                            Error = 1/2 * ∑(ti - yi)²\n",
    "误差平方和越小，说明拟合度越高，越接近于真实情况。"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在求出误差后，我们需要将误差反向传递，修改每个节点间的权重，我们运用随机梯度下降算法对目标函数进行优化：\n",
    "\n",
    "                                        \n",
    "                                            Wji ← Wji−η∂Ed/∂Wji\n",
    "\n",
    "根据链式求导法则，可以得到：\n",
    "\n",
    "                                            ∂Ed/∂Wji = ∂Ed/∂netj * ∂netj/∂Wji\n",
    "                                                  = ∂Ed/∂netj * ∑∂Wji*Xji/∂Wji\n",
    "                                                  = ∂Ed/∂netj * Xji"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里权重的推导需要区分从输出层到隐含层的权重和隐含层到输入层的权重，对应链式求导有不同。\n",
    "对于输出层到隐含层：\n",
    "                                    \n",
    "                                            ∂Ed/∂netj = ∂Ed/∂yj * ∂yj/∂netji\n",
    "其中yj为netj的函数，即yj=sigmoid(netj)，并且有yj的一阶导数为：yj·(1-yj)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于隐含层到输入层：\n",
    "                                        \n",
    "                                            ∂Ed/∂netj = ∑∂Ed/∂netk * ∂netk/∂netj"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上就是神经网络的大概流程：输入→隐含层→输出→计算误差（均方差）→误差函数反向传递→计算各层之间的权重更新值（运用梯度下降）→反向传递，如此重复训练，设定误差范围，最终能够找到最优的拟合或分割曲线。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BP神经网络实现Iris数据集分类"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "error= 0.1057218343740031\n",
      "测试分数为: 0.9333333333333333\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEICAYAAABVv+9nAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABMaUlEQVR4nO3dd3yV5fn48c99ZiaEQIAk7E3YU/YUERVbq6JVcevXqv212n5rqx3Wtrb91mprWydu6sA9cSAgW/YeYY8EBEIgO2c89++P5yQkOSd7nHW9X6+8SJ7znOe5EpLr3Oce16201gghhIg8lmAHIIQQonlIghdCiAglCV4IISKUJHghhIhQkuCFECJCSYIXIkQopazBjkFEFknwQjSAUuoHSqlk3+fTlVKP1OE5FyilHqjw9ftKqd/6Po8DNiilZjZb0CLqSIIXop6UUt2Bt4BJvkNZwANKqf61PLUAuE8p9Xvf16VAke/zp3xfr27icEUUkwQvwopS6mallPZ9eJVSB5VSP/U9trTCY4VKqeVKqZFVnje9wrVeVkotrfDcgxW7SXzn3xwgjAeBb7XWHwBorXcD84FXlVK2ADHblFKxwC5gNjBFKeWs8HhPYJjvsUKlVLxSSjXm5yQESIIX4ekM0AZoB/wIeFQpdbHvsX/7HhsIrAc+VUq1rvDcX9Rw3W7AnJpurJSaCNwC/KzKQw8AnYFnAiTnq4BCIA/4ChiC2Zq/BvgDsAHoCmQCub7vL7mmOISoC0nwIhxprfVZrXWu1vpzYAcw1vdYqe+xg8DPgaQKj3mBi5RSQ6q5rhf43+pu6nuheAl4Rmv9bZWATgM/AK4F3q7yorJAa23RWidorZOAm4CTwCbgN8DjwKfAIK11K621U2udU6efhBA1kAQvwppSahiQAWwO8LCBmbTtvq/PAouovhX/PjBIKTUjwH1igHcBD/CcUqq7UqpbxQ8gG7gVmAhsV0pdAqC1NnzX6KaUegl4DLgEOOSL7QnMhL9dKfUrpZQdIZqAJHgRjpKVUmeVUnnARuBFrfX7FU/w9YX/FDMhr6nw0F+AOUqprgGuexh4k8AvAL8B+gGPAFuAA8DBAB8vAiOAPcB+XyytlFILgbW+5w3WWm/B7KY5p7XO11rfB1wE3AH8qV4/DSGq4TcgJEQYOAsMx2z9jgP+pZQqS+L3KqVuBxIwZ7fM0VqfKusW11p/rZTaBNxfzbX/Cmz1vTOo6LeYM12OA29j9tdnAu211qcAlFI3Ag9orY8BF1Z4bhFmq/1brXVBheOFwDTfddFaf6uUGgoYSimr1tpb55+IEAFIghfhyNBaH/J9vlcpNRpz0DMPeBV4FCjw9YsH8lfgFWBx1Qe01tuVUp9SpRXvS7ZZZff3zc45UpbcfToDRwLc73fAL4FSpZSnwvFWvmudrXDMBsQAt/liFKLBpItGRALF+X72PK31oRqSO5h97VmY/eCB/BW4upZ73gJ8WOVYwASvtf6N1tpeNsjqG2i9AfjOF8fdZcd959i01pLcRaNJghfhSCmlkpRSHZRS38NMll/X9cm+Qc+/AQFLA2itV1C5377qze8HxmB2u1TUhcAt+KrPnwA8B/w/zBeZvymlfiGlCkRTkwQvwlEy5nzxLOBJ4FnMxUf18SrmrJfq/KXqAaVUO6XU05gDrVdqrY/4jg/xzcMfD2wNdDGllFMpNVUpNR9zSuTPtNZva613YI4jXAXsVErd51spK0SjKdmyT4jaKaU6Yi5I2g/8yJeYyx77G3Al8AbwO621p8pzr8Zc6Zrj+/cxrfXJKudYMOfH/wJzAHeg1np/s31DIipIgheijpRSnXwzZOr7PBtmKYL1ug5/cEqprlrrww2JUYiKJMELIUSEkj54IYSIUJLghRAiQoXMQiel2mlzbCk0KAyGsYlANVsNFJsY3uIxiabVp8+GYIcg6iA/PzXYIYS048ePn9ZapwR6LGT64JUaqc3qrqFCc5DudMN/rGs1Yxgn+zKEvSVLpOR6KPvmm4eDHUJYePjhhzdorUcGeky6aKql+An/oJC48iMGUEgcP+PvwQtLCCHqSBJ8DT7i+1zGJyxlEtmk8jkXM5lvWM24YIcmhBC1Cpk++FC1lKksZWqwwxBCiHqTFrwQQkQoacGLqCODq6FPBlibhrTghRAiQkmCF0KICCVdNEKIkCFdM01LWvBCCBGhpAUvoooMsIpQkpeXwNq1o8nKSqdDhxNccMFa2rQ512TXlwQvhAi6aOyaOXWqLfPm3Y7HY8PrtXH4cBc2bhzBzTe/Qlra8Sa5h3TRCCFEEHz++cWUljrwes12tmHYcLmcfPLJpU12D0nwQggRBIcOdSNQCs7OTsPrbZquREnwQggRBA6HK+Bxm82DxdI0VX4lwQshRBCMHLkem81d6ZjN5mbIkC2oJpoLIAleCBFU0TjACjB58jf06ZOJzebG6SzBZnPTtethZs78osnuIbNoRFSQ6ZEi1NhsBnPmvENubhKnTqXQtm0Obdueadp7NOnVhBBC1EubNmdp0+Zss1xbErwQIiiitWumJUkfvBBCRChJ8EIIEaGki0ZENBlcDT3SNdNypAUvhBARShK8EEJEqAYneKVUB6XU8hoetyulPlFKrVJK3drQ+wghhGiYBiV4pVQb4BUgvobTfgys11qPAy5TSiU25F5CCCEapqEteC9wDZBXwzlTgAW+z1cBIxt4LyFEhJAB1pbVoFk0Wus8AFVzRZx4IMv3eR7QoeoJSqk7gTvNr7o0JBQhqiUzaES0a85B1gIg1vd5QqB7aa2f01qP1FqPhJRmDEUIIaJPc86D3wBMAN4BhgBrmvFeQogQJl0zwdEkCV4pNQ3I0Fr/u8LhV4DPlFITgQzg26a4lxBCiLppVBeN1nqK79/FVZI7WuvDwAxgJXCh1trbmHsJIYSon2YtVaC1zub8TBohWoQMroYO6ZoJLlnJ2iQ0o/mWy/iYdpwKdjBCCAFIsbFG68JhvmIGqRzHixUnpfwfv+Bhfh/s0IQQUU5a8I30MbPpwX4SKSCJc8RSws/4O5fxcbBDE0JEOUnwjdCX3fRkPzaMSscTKOTHPBmkqIQQwiQJvhFakYenml6uZHJbOBohQosMsAZf1PXBp3CS6XxNIfF8yUWUEtPga21hCArtd7yIGN7lB40JUzSAzJ4RorKoasH/hCc4TFee5X94jbkcpyNjWdXg67lwchdPU0gcHt+PspA4jtKF/3BvU4UthBANEjUt+BGs50/8mlhKiKWk/PinXEJHvsOFs0HXfYPr2ckA7uXfpJPFp1zKS9xCUY2VlIWIXNI1EzqiJsHfygs4KyT2MhYMLuJLPmF2g6+9haHcwbzGhCeECCFer8LttuN0uqi5aG5oi5oEn0iB32wXAAXEU9jyAQkhQo7HY+Grr2awYcMIDMNCYmI+l1yykL59M4MdWoNETR/8u1xJfoBuEztuFnFhECISIrJ8883DYd898+mnl7Jhw3A8HjuGYeXcuSTeeedKjhzpFOzQGiRqEvzHzGYpU8gnAQAPFgqJ4wH+Qg7tghydaCyZQSMaq7jYydatg/F4HJWOu912li2bFKSoGidqumgMrHyPj7iMT7iKd8ijFS9yK5sYHuzQhBAhID+/FVarB6+3alpU5OS0DUpMjRU1CR5AY+FjLudjLg92KEKIEJOUdBat/Ts1lDJIS8sOQkSNFzVdNEKI5hPufe8ADoebceNWYre7KhzV2GweJk/+JmhxNUZUteCFEKImU6YsIzExn5UrJ1BYGE96ehYXXfQV7dufDnZoDSIJXogIVmSFXAeklILDf5awqEIpGDlyEyNHbqr2HMOAM2faYre7ad06rwWjqz9J8CKsyeyZwDwK/tULPu8IVl+5pLmH4dqj5toP0TD793fn/fevoLTUidaKlJRTXHPNApKSzgU7tICkD16ICPRsD/iiI7isUGwzP17tBl92aNr7RMLc97rKzU3izTevpaAgEbfbgcdj58SJDrz88k0YIfruSBJ8BX3Yw7W8wQWsgQBVIoUIBx4Fn6RBqbXy8RIrzO8anJgiwfr1I/B6K6dMra0UFcVx+HC34ARVC+miAWy4eZNrmcVCPNhQGBygJ9P5WhZBibBTYjWTfCBnHIGPi9qdO9cawwicMvPzE1s4mrqRFjzwMx7jYhYSRzGtyCeRQvqzi5e5OdihCVFv8R5o7Q7wgIZ+oT0mGNJ69DhYZQqlyTAsdOp0LAgR1U5a8MCPeIZ4iisdc+DmIr4kngIKfeUNROiQwdXqKeDHe+HP/c930ygNTgP+50DT3CNa+t0rGjRoGytXjuPs2aTy1a52u4sBA3aQnByaO7hFYYLXTOYbruJt3DiYzw3EUlTNmQonpZLgRdiZfBpabYNXukJ2LPTJh1sOQU8pnNpgdruHO+6Yx8qV49i5MwOHw8WoUesYOnRzsEOrVpQleM1z3MkPeYM4itAo7uQ5dtGPNpzFjqfS2fvpyRnCswaFEMPOmh+i6cTElDJ9+hKmT18S7FDqJKoS/DhW8UNeJ6G8xa6Jp4ihbEYBBgoLmhIcuHFwCy8FM1whQlI0ds+Eq6hK8N/jA+Kq9LUDWDHKF394sHCQ7sxgEVmEZw1oIYSAKJtFU0wcXqx+xysO19kw6MoREslvucCEEKIZRFWCf53rcGOv9TwvVkayvgUiEg0hM2hEuDpxogOffXYx7733fXbt6odhNO/vclR10WTSl/t5nCe4Dw82HLhw4PKrzaGBQ3QLQoRChC7pe2+cdetG8MUXM/F6rWhtYdeu/nTpcoTrr38di6V5Vs5HVQse4FnuoiuHuYf/cB+PU1Bln1YXNo7RmRVMCFKEQohIU1wcwxdfzMTjsZdvKuJ2OzhypDO7dvVrtvtGXYIHOEV7XuNGnuYeLuRr9tMdN1Y8WNjMMKbxNVJzT0QLrwJvsIOIcAcPdsNq9f8pu91OduwY0Gz3jcoEX1F/dpHKCbRvimQGO/kzDyLFxkSkO+GE/x0MF02CmZPgoYFSq6a5OByBakcAGDidpc123wYneKXUC0qpVUqpX1fzuE0pdUQptdT3MajhYTaPNpzhaX5EHMU48GBBk0AhV/M20/k62OEJ0WxKLHDPcNiYBIYCrwW+TYZ7hpkt+oqiqSRwc+nW7SAWi39NYbvdw/DhG5vtvg1K8EqpHwBWrfU4IE0p1TvAaYOBN7TWU3wf2xoTaHO4iC8DzqqJo5Af8kYQIhI1WbJERdUMGg24VPO8l1yaAsVWMCpkAK8F8uywOrkZbhjlbDaD669/nZiYYhyOEhyOUmw2N5Mnf0PnzlnNd98GPm8KsMD3+WJgArC3yjljgCuUUuOBw8BNWmsPIcSNPeAfj0ZRirxXFcFhAP/tAm91Mbfca18Cd++HSU24LejROHMTkKpcFjgWB+Q03b2a0vHjHdi+fQCgGDBgB2lpJ4IdUp116pTFz3/+d/bv70lpqYPu3Q+SmGgWB3K57GgNTmd1XTkN09AEHw+UvezkAb0CnLMOmKy1Pq6U+g9wCfBRxROUUncCd5pfdWlgKA33BTOx4v+2qYRYXuPGFo9HCICXusM7ncy67gDfxcKj/SFuO4xsoqKFPQsh1uOf5O0GdPcVJAu1bpmlSyexYsUE3zRDWLt2NBdcsIYLLwyPujAANpuXvn0zy78+d64V77//fY4cMfNfWlo23//+B7Rrd6ZJ7tfQPvgCINb3eUI119mqtT7u+3w34NeNo7V+Tms9Ums9ElIaGErDFZLAD3mdUhy+WTRWSnHwGD9jDWNbPB4hXKpyci9TaoWXujXdfSacMmvGWyu0b2wGdCiBUU2TW5pUTk4yK1ZMqDDN0ILb7WDNmrGcPNnyuaMpeL0WXnjhVg4f7oJhWDEMK8eOpfPii7dSWlr7gsy6aGiC3wDlE8WHAIcCnPOaUmqIUsoKXAFsaeC9mo0dF7/kL3iwYceLBS9erBSXv3YJ0bLO2avvc89qwl9Lh4anNsK0kxDjgTgPzDwB/9ocmlPr9uzpi9b+4y9er4Xdu/sGIaLG27OnDyUlMWhd8dXcgtttY/v2gU1yj4Z20XwALFdKpQGzgGuVUn/UWlecUfMI8DrmhPKPtNaLGhVpM5jDAoawlXhfdUkLEEcxv+MRXuQ2TtE+uAGKqNPGDdZqMnz3etRyN6g9Ubdxw4O7637NYLJYvCjl/4NRSmO1huiO17XIzW2Dx+NfG8vtdnLmTJsmuUeDXqy11nmYA61rgKla6y1Vkjta6+1a68Fa60Fa64caH2rT+z4fkID/X40LOxNZXs2zNN05QGeONG9wolw0zZ6xaZh7CGKqrIlxeuG2gzU/1wDmd4HZ42H6ZLhpFKxrYJ4Itf73jIxdAY9bLJqMjB0tHE3TSE09gc3mv/jJ4ShtssHjBr8b01rnaq0XaK3DZxi7itO0w1PNj+Acrf2OjWIt++nJdgayh75sYTB92NPcYYooc80xuHcvdCg2Bz375sFftsLAWvZTfb4HzO8KBXZAwZF4+M1A2NaqRcJuVq1a5XPZZZ9gs7mx213Y7S5sNjezZi2kTZtzwQ6vQbp3P0i7dqexWs/PnLFaPSQm5tOvX9O8tVJah8aKTaVGalq4guMwNrKcCZX2YzUwSxmkk4W3Qg9WW05zgB60qlBG2IviNCl04QgunC0ZelSJltZ7Y5RY4Pvjz+/BWk7D8Fz4+9b6XS/UWvBlCgriyMw0++P79NlTPs2wsbSGkhInDocLa3V9ZM2gtNTO0qVT2Lp1MForMjJ2Mn36YmJjS+p8jYcffniDOVHFX1RVk6yqDbkYWMoHtTSK07TjQhZVSu4A1zMfW5Ut/axoYilmNh/zLle1UNRC+MtxmBtr+1FwJK7Fw6nVkSOd+fzzmXz3XQdiY4sZP34FY8asRdXyWp6QUMTw4ZuaNJbt2zP4/POZFBXFY7V6GTVqHdOnf90iid7pdDNz5lfMnPlVs1w/ahN8Ktl8xOXlA6wAGo0VD3v9Z3TSmaMBd4Ny4CKN7GaNVYjatHNBgEkmoKFHPRq5LdFyz87uyGuv3YDbbS4mLChIZPHi6RQVxbf4Xqf793fnww+/Vx6LYVhZt26UbyHSIZTS9Oq1H6fT1aJxNZVQnBHVIubyKpYqNfQsgB0Ps/nY7/yVTCCfBL/jHmysYUxzhRn1pHumbpwGzDkaYHDWgJsPBSWkai1dOgW3u3Lb0u12sHr1WFyulm1zmrFUXrXudjvYsGEkH344mw8/vJzHHvsZu3aF51TMiE7wTkp4gD+zi77spjeP8xPGshIbblI5Tiz+VdxsuGnPSb/jHzObTHpTREz5sULi+IbJrGN0s34fQtTFLYfgtgOQXGouYOqdbw7O9g+x3Se/+64DgVKPxWKQl2dObsjLS+TTTy/m3/++m1dfvYH9+7s3Syy5udVNM1K43TG4XDG43Q7effdKCgpCsK+rFhHcRaP5kosYyfryrpU+PMlP+BcGFs6QRCkOnLiqPEvxDZP9rubFxiSWcz+Pcz3z8WDnee7gKe5uke9GiNoo4Kos86O+WnJQtV2705w715qqey4YhoXExDzOnUvkmWfuorTUgWHYOH06haNHO3PxxZ8zYkTT9r+nph5n7954amvrKqXZtSuDUaPCayvPiG3BT2UJw9hUqd9cARY0Nry0JwcHLowKv2QFxPMuV7KDwKvIiojnj/yG/uxhENt5kp/gqcMer0KI86ZMWYrNVrmolt3uYuTI9TidbpYvn0hpqRPDON/+dLsdvh2RzGlCubmtyczsRU5O40pfTpu2BLu9ag1E/8FVr9eCyxV+f+sR24K/gG+JDTAoWpGZ2jWH6cxhuvEcd/I617VEeEJErc6ds7j22rdYuHAWOTltcTpLGTNmDZMnLwPgwIEeGIb/Ck+AkyfbsWLFRDIz+2C1evF6rXTrdog5cxbgcNS/WG1q6gluvvkVvvrqQo4fTyUmpoT8/IRKLy5gLqjq3btqwdzQF7EJPot0ioklkYIaz1OYLfPJLGuZwIQQ9Op1gB//+D94vQqLRVeaHpmYmM+ZM239nuP1WtmyZQiZmX3weOx4PGaL+tChbnzxxUxmz/60/FytqXXKZZn09GxuvvnV8q8/+eQStmwZgtttBzR2u4eRI9fRvn0T1mtuIRGb4N/lSp7gvjrV5Ki68bYIPpk903KCuagp0Fzz8eNXkZ2dVml2i9XqoUePA2zdOqQ8sZfxeOxs2TKESy/9lLVrR7N8+UQKC+Np2zaHmTO/pE+f+rW8L730MzIydrJt20CUgsGDt9KtW3iWJonYPvgi4pnEMnYyADfWaiv0aeAv/LIlQxNC1KBPn71Mn74Iu92F01mCzeame/eDXHnle7hcgTfi8XhsLF8+ga+/nk5hYQKgyMlpx4IFV3PgQLd63V8p6NHjEN/73idcfvknIZ3cS0pq3pgoYlvwADsZwCC204kjzOYTHuJPpJGN5vz4/Ztcw3tcGcwwhWh2B+Jhd6JZ733Y2fq37IqLneTnJ9KmzdkAg5JNb8yYdYwYsYnTp9sRH19Aq1ZmV2vXroc4cKAHVb+D1NRsVq6c4Den3eOxs3jxNHr0eLHZY25JBQXxvPfe9zl8uBvwYLXnRXSCL3OMLjzN3TzN3XTmMD/hSeLJp5B4MtjN37mf/3AvB+gZ7FCFqFWh1SwpHFOHKrkeBb8bABvamI0apc0ywf/YBCl1WJzp8Vj45JPL2LZtEFarF60VEycuZ+LEFXXu424ou91DamrlWoazZn3OvHm34/HY8HptWCwebDYv06cv5s03rw14nZwc//78cKY1vPzyTZw5k1ztYHSZqEjwFR2lK09wHxsYQSJ5xFHCdL7mTp5nFgtZwcRghyhEQPvj4a/9zNa4Akbkwi92Q3IN23i+08lM7hWLkJVa4E/94YonH671nl98MZPt2wfg9ZoJFWD58om0apXH0KH1rGDWBFJScrjnnqf49tvRZGWl0bHjd1xwwbe0apWP1er1658HaNs2RDeYbaAjR7qQl9eq1uQOUZjgAf7EgySTg91XqsCBGwduXuBW+pJJ1QUYouXI4Gpg5+zwk2Fm673s13N9G/jpMHh5bfVdLh+l+VeYNCywszVcHFNMbEn120R5PFY2bRrmlzTdbgfLl09ssgS/detAvvlmMnl5rUhJOcWMGV/Rvfvhas9v1SqfGTO+9js+ceJyvvlmcqVuGpvNxfTpi5skzlBx9mxSnc+N2EHWmsxiYXlyr6grR2hbZTv5wWzh+7xPdw60VHgiyuTZYEtryI6p/pyFHcCtqNT28FrgtAM2J1X/PHcNf+Eea8196S6XI+A2eQCFhU0z82z9+uF8/PFscnLa4XY7yM5O57//vZ5Dh7rW+1rjx69ixoyvSEjIRymDlJSTXHPNArp3P9QksYaK1NRsDKNuqTsqW/D5JNKewHNay/Zjbc1ZFjKLwWzFjQ0nLj7kcm7gv36lhIVoCA3M6252o9i1mcAH5MEj2yGhSvvjSDy4Arwj18DxGl4YJp6Cj9PAUyUftDrTlsTCxBrji40tJja2iIKCqjuGaDp1Olbjcw0DMjP7sGtXf5zOUoYN20Rq6nd+53z99fSAA6OLFk3n9tvNgdGiohhyc5NJSsolPr76xYtKwejR6xk9OrzKCdRX+/an6dVrH/v29cTjqXkWTVS24P/D3ZWKhgGU4uAzLqHINyf+Oe5kOBuJp4gk8oilhNl8ws/5WzBCjgrR1j3zVQd4r5OZuAtt5r/bW8Gf+/uf2z/P3Bzbj4JeNazlu+kQpJRCrO+5Di84Sh1c8f4VtcanFFxyyULsdleFYwYOh4sLL6x+i2XDULzxxg95770fsGXLUNatG8ULL9zK2rWV96QoKYmpdtrjqVMpGIbik08u4fHH7+fVV+fyxBP38cEHl+P1RtfvSSBXX/0Okycvo3Xr3BrPi7oEP4Mv+SX/hx0PGvBioZBYNjGMWzFbDE5K+B4f+hUii6eIe3gqCFGLSPRWZyip0ip3W2FdMhRUeZN44XeQ6DGrRJZxeCHjHPStIcG39sCL6+DefXDRCbjhMPz4Xz+mU3anOsWYkbGb669/nR499pOUlEtGxg7uuGMeHTv6V1wts2dPXw4f7orLZe5yprUFj8fBl19eRFHR+T5/p7MUq9W/qxQgKeksK1aMZ8sWc2FTaWkMHo+d7dsHsGTJ1DrFHsmsVoOJE1dy331P1nheVPU1dOMg73NFpU0+QJNHK8azEgPzr81JKRYCz0FLqKX0gai/aGq55zjg6/Zmv3tONbWrLBoKrJBQocUea8AzG8wunZUpYDPg0uNmwq5NjAGXnDA/GrJqtVu3w3TrVocb+ezYkVGe3CuyWLwcPNidAQN2AuYq1vHjV7JiReX563a7i6lTl/DRR5cH6L5xsHbtaC68MLIGTptLyLTgB7CDb5jE9/ig2e5xB89jo/KcMiuaOIqYirmTTAzFXMqnnMG/Sp0HKwuZ1Wzxici2Jhmuv8BM0v8t2xw7wBLrOC+099+qgGQ3/CITPlwJ766GWw+BIzS2VK7E6SxFKf8GklJgt1f++xs9ei3duh3AYvGglEF8fB6XXfYx/fplUlISeHDB5XJg1GENgAihFnwMJUxiOcPZyF95gD/ymya/R1cO48R/0rAFTSrH6ck+VjKeWIqJoxCN+fdnAYqIoZAEfslfmjyuaBVNLXeXgj9kVJ6y6FWANrtdvBZQhpmw798TQi2vBhg2bDNbtw72a30rpenR4/xstNOnk3nhhdvweGwYhg273dzwukePgwCkpWVx7FgXv+t36PAdlnD+AbWgkPsxJVDIgzxKa8422TV7sJ9H+A0dOEEx/q0CGx5WMY753EA7TtOKfGy+SvFerOyjB3/k1/RjN0fx/4UTdbdkiSr/iCbbWlezukKZrfV+eTD1FPxzE0wI43U5RUUxLF06Ca/XCmiU8mK3l+B0lnDdda9js53vc//oo8spLo4pfyFwux0UFCTw5ZczAJg16wvsdhdKmc9RysBud3HJJQtb/PsKVyHTgq+oFCfD2MRSGj+YcjVv8TK3YMODAzdeFF4sWH197AXE8ybXcoZkhrGp/HgZO15iKOXPPATAdBbxC/6PdLJYxIX8lQc4Tlqj44xU0ZbIq2Omu8C6F8KftrdkNM3D7bbyxBP3+crs+nZb0Bbsdi8//ekTOBznk7vHY+Xo0c5UbWMahpU9e/oBZhnfO+98nuXLJ3D8eCodOnzHxIkr6NCh+gFeUVlIJng7bo6T2ujrxFHIi9xWaVcnK5pSbGSTRjZp/Id7mM8NJNXwjkH5/jTv4Fme4P7yQdpe7OU6Xmcom8kmvdHxRgpJ6v4GnjPrx1QV44VZx5v//i1REnjx4mmVkrtJUVQUx969fRgwYNf5o0qjlEYH+JlUnFmTknKaH/zgg2aLOdKFXBeNCzvbGcAe+jX6WhNZjhf/1SFO3GxmCGP4lte4EY2FXJLZwmC8Vd5IF+PkNW7ASQmP8b+VZuA4cdOaczzAXxsda7iL1q6XurJpcwFTrMecz24zwOmFad/B+AZ0yXiV2e2zrZWvL7+FFRbGsWbNBXz11XT27u3pW9jUm+rKfOze3bfS11arQe/ee7FYvFWOuxk8eEtzhR11QqYF78VCMQ7WMoqreadJrlmK/1StMmUrViuay3xWMIEYSoingAIS2E8v/sSv6cuegNdx4OYivmySeMORJPS6G3oOFqyGZSnmPPeRudCjsP7X2ZQEDw8wK0WC+WLx+x3m9VvC4cOdmT//et/8djvr1o0iNfU4sbFFUKkY93lJSf7BzZ79MS+9dAv5+YkYhgWLxaB9+5NMn76k+b+JKBEyCX4P/ejNF2RRtwUYdbGcibgDbIpdQDzzuMPveCZ96cYhruIdunGI9Yzkcy7GwMpJ2uMgcH3VrCjsg5fE3jAJXnM+ekOds8ODg/wXSP1qsPnikVhltWtTd80YBrz99tW43ecbTy6Xk+zsNIYO3RBw1gtoJkxY7nc0IaGIe+75DwcP9iAnJ5mOHb+jc+ejzV6GOJqETIIvJrZJkzuAFxuz+ZjPuRiFxoIXKwZP8yO+YkbA5xQRz6vc5Hf8BKksZQpTWVJphWsBcfyNXzRp3KFOknvwLEkJPFirgaUpMLuZ+/NPnuwQsLyA2+3g2LEujB69hrVrx1R6bOTI9bjdDpxO/1oLFgv07HmAnj2lmF9zCJkE31zWMJZUjnMZn9Cac3zNdA7So0HXupY3eYtrmMQyXNixoPkVf+YLLm7iqIUILN8OrgAjZx4FedWsjG1KShnVVphUyiAzsx8Wi1GhVrlm/fqRbNo0jBkzFjFmzNrmD1KUi/gED1BMHG8zp/zr/uzkQR5lGJvYyiAe5SG2M6jW65wjiYv5gjSyaM9JdtOPkgB9+ZFKWu7151Wwui2sawPJLrj4BHQIsEq1roblwutd/Lto7BqGV6k71RwzZ9q3P0V8fCFnz1aeLWO3u0hLO8HWrYOrbERh1jj2ei0sWnQhXbse9qsqKZpPyM2iaW6jWMtaRnEtbzCAnczhbVYzhvGsqPM1sklnM8OiKrmL+nMp+MlQeLQffJQO/+0CN42Gtf5VMOpsQB6MyjGnV5aJ8cCYHOiX3+iQa6UUXHPNAmJiSnA4SrFaPdjtLnr23Ed8fGG11SEBvF4rmzcPbf4gRbmoaMFX9CT/j4QKUx2tGCRQxL+5l2FsDl5gIuJ8lgr7Es6XJ3D7/v1jf3hvlTl1sr4U8LudsLQ9LOxofj3rBEw52XL7kKWmnuD++59g9+5+FBTE07XrYdLTj7N9ewYOhytgoTEwFz25XC3QjyTKNTjBK6VeAPoDn2mt/9jQc1raCDYEPD6YrSgMdPS9qRHNZFEH/+3ywOy2yUyAjAa2uK3A9JPmR7A4HG4GD95W6Vi/frv56qsZuN02tPb/xu32UjIydjdLPGfOtOHTTy/hwIEeWK1eBg3axsyZXxATU4edxSNYg7KZUuoHgFVrPQ5IU0r1bsg5wXCWpIDH80msMbl34TBzeZVL+NSvImWkkwVMDWOvpuKhpuYqkKUWOBLnXxM+1NlsBrffPo/+/Xf7qkmWleszk3uvXvvp2XNfk9+3uDiG55+/nQMHepTPzd+6dTCvvTY34ErZaNLQX6EpwALf54uBCcDeBpzT4p7gpzzEo5VWpBYSx7+5t5pnaP7Gz7mHp/Bgw8BCKU6m83WdBmZF9JqdDbsToaTiX5mGVh7oGWBbAQ280Rle6wZKmzNjpp+E+zPNQdS6aomyBNVJTCxkzpx30BqOH+/I5s1DcLsdZGTsomfPfQ2uAllUFMuOHRkUF8fSo8cB0tOzy+fLb9o0FI/HhtbnL+712jh5sj3HjqXTuXNWE3xn4amhCT4eKPup5QG9GnKOUupO4E7zq5ap0vhXfkknsriFFynFiRMXr/NDfsfvA54/m4+5i2eJpaT8mAF8yqV045B06YhqTT0FG9qYXTUKsxaNTcOftgXuL1/U3kzuFWfILGkPDgPuC3rTqH6UgrS0E6SlNWJVl8/Bg914/fUfAmaRsuXLJ9KnTyZXXvkuFgucONHRrzSxSXP6dEpUJ/iGZqcCKJ9CklDNdWo9R2v9nNZ6pNZ6JKQ0MJT6MbByD0+RTjYX8jWdOMadzKt2I+27eIYEKq8ntwBtyGU4G1sgYhEqii2wOAU+TYUTNWx0XUYB/5sJz2+Ae/bBA7vh7VXQs5ryBP/t6j/9sdQKn3cMPPc9Gni9Ft56aw5utwO324HWVtxuB5mZfdi5MwOA1NTj2GyB+9pTUk61ZLghp6Et+A2YXS5rgCEQsFBLXc5pcnZcpJPFKVIoJKHa83JJZkOAXZuqiifwX6OBhbhKW/+JSLatNfxykK9XWZnv4uYchdsO1f7cLkXmR23OVDPDUCsospot+ZoEs2umuRw92gnD8H+/43Y72Lx5KAMH7mTo0C0sWzYJr/d8N43V6qZDh5Okp0dv6x0a3oL/AJirlHocmAPsUEpVnSVT9ZxPGxpkXd3Lk5wihW0M4hQpPM1djR4QfYNrKSTO77hCs5bRjbp2OJDBVXM++4MDocgGxTazle2ywjudYXNS090nI8/se68q0Q2tqvwaa+DjVLhmDMyYBHeOgINdDzZdMGEkNraEO+6YR48e+7FYvNjtLoYM2crcufOjvq5Ng1rwWus8pdQUYAbwf1rrE8CWWs5p1lp3V/E2f+FXlQZP5/Iqbuz8P/4FgBUXI9jIHvpwrg6td4CXuJWbeJWBbCORQlzY8WDz9eHX4X26CHubkyBAI5JSC3zWEYaeNVv069vAoXjoXASjzxCgUHXN7jwAW1ub1zV8TS+nF+7d598Se7MzvNrtfJfO3kQ4dN3rzH1tLl0CFvwKT507H8Ni8X/Vs9tLGTr0fMpJTs5l7tzXWzK0sNDgnj2tda7WeoEvuTf4nKbya/5QKbkDxFPMbbyAkxIWMRU3TtYwllzakk1HYgkwlaEKF04mspxbeYlXuJEn+ClD2MJ7XNVc34oIMe5q/kq0MpNxvg1uG2WW8H2+h7n36s2j4Ww91/T0KISnN8Lk09Cx2Cw98Jdt5mBtRR4F8wP017sdbhZPW1zpWFFRDAcOdOfkyZYZ42pqVqvBnDkLsNtdvu37zG37+vTZS0bGjmCHF/LCbKZt9dLJDnhcofkv1zGNpZVmLnTkOw7Qg1RqXy3ixcY7XM07XN1E0YpwMvRs4E01Yjww7SQ81ROOxYLH90LgscBxC/yjNzy8s3736loEv63ynF2J8EVHs6toyinoVlj9Jh8n25//fV6yZBIrV07AavViGBbatTvN9de/TkJCA4rQB1GPHoe4775/sGPHgArTJLOivvulLsI+wbfhDD/kDc7RmmRy/N6S5JPI9/nAb1qaAjpwio5kcYJ0QDOS9QxkO5n0YRXjaLnF3yKUxXvh/j3weF+z9exVZi2Y4Wdhwmn4S//zyb2M1wIr21W3/UXdze9ittZdynzHsKQ9jK1hB6i2OW0B2LWrH6tWjcfjsePxmG8lvvuuA2+9dTW33fZyIyIKjri4YkaNWh/sMMJOWCf4kaxjEdOx4SGe4vJ1c2V/UIXE8TP+zqvcWO01hrCFPJL4nIsZxiYANIq99GYaizlXzcpXEV0uOgn98+HLDmbJ3vGnYUSu2cdZ3eQWQzUuwZ90wmtdzQHdMiU2szrlxFOwIqVyN43NZWPqUnOj+tWrx/jNDTcMK8ePp3HuXCuUMli4cBaZmX0A6N9/F7NmLSQ+vhgROcJ4dq3mLa6hNfnE+zbVVphb/52jFSsZx5W8y3zm4sJe7Y72y5jEo/yKkawjgUISKCSRAgawg/9wT4t9N6FGyhP461xsTov86V4YlXv+j2dcDlirZHmLAaPONO4PbG1y4OeXWs2ZNTcchgS3OfMm6UwbJj3zI5wrxmMYUFTkP/MLwGIxyM+PZ96829m9uy9erw2v18bOnf158cVbA05JFOErbFvwPdlPB/zrStswyCKNCawEoC+7yaQPA6k8IKOB9QynmARu5FViqVyk24mLq3iHubwmq1VFje7dBztambVjim3mxtqxXrgvs3HXdXoDT5u0aIg14PojcN0RWLL8e/z1j2+xUmlWaEVMTAndux8gN7cNXm/lP3GLxeD06RRKSmIqFQQzDBv5+Yns3duLvn3DbNmsqFbYJngvVlQ17fKyVan92MVaRhNHUfnb5LJnfMKlXM4nAJW24KvIhgcLBl5J8KIGbV0wfy18kwIH4s2B0iknIaaWhUm1GZcDjwdoUNs0zPS1bd5/70mefvqu8n52AJfLwfbtg/B6rZR1EillYLN5uOSST33b7vmX9PV4bJw+ndJkCd7rVaxZM4Z160bidtvp128PU6cuDbtB3nAWtgn+EN05RDf6sRtLhURfSBzzuB2AP/Ab4ijEWuFxBZwhiR/wHjP5nAmsYA99GMQ2bBV6U70oVjCh2hIGQlTkMGBGE29UFO+FP2yH3w4wf2815gDvvfvOr4zduHEYhlG1AaJ8yf18s0Ypgzlz3qJ37wNs2WLB4Sj1S/I2m4d27Zpuaf+77/6AzMw+eDzmWMCmTUPJzOzDPff8J+rL+LaUsM5eV/EO3zAZJ6U4KcWNneVM5CnuBmAsqysl9zJOXKxkHP3ZQwIFFBODBU0xMcRSQiGxuHByJ8+19LckQlSezVzIZNNm33psI1vndTUyF95dBeuSzZk6I85A6wp7VxcUJFTZIq+MqvS5xaLJzk7j1Kn2rFw5zrfz0vkhYIvFQ0JCAb17N00539On25KZ2bfSOwvDsFFSEsPmzUMYM2YdhgEHD3bn7Nkk0tKOk5ra7Mtlok5YJ/hdZNCFI3yfD+jMEWIpphPHeIC/8BK3coz0gPPjHbgYyA7ifBUiy/4tIpYFzGErg3iZWzhD2xb9fkKBDKz6W9gB/tHHrAapMGfN/H4HjM6t7ZlNI9aASacDP9anz1527epf7S5KZcpqpOflta4yu0ZjsRhkZJizaAKtGq2Oy2Xn66+nsXnzEAzDSq9e+7j44i9o3TqP48dTsVi8QOXVXm63g8OHu5KRsYuXXrqFgoJ4DMOKYViw211MmLCCMWO+xeHwBL6pqJewTvAAJcTyCZexmrF05TCJFODByh/4LQYKLxasFbpeioilBCfJnPW7VgylPMqDZNK3Bb8DEcqOxsI/+1Seqgjwu4FmZcgEb+DntZR+/XaxevUFfPddxZK5/pMzbTYXubnJAVr7ml69MrnqqvfqdV+tYf7868nKSsPrNZP47t39OHKkCz/+8b9o3fqsXwwAVquHtm1zeO+9Kzh7tnWlgV6XK4alS6eyc+cAbr99HjZbC71NimARMXr4Mx6jJ/tJ9JUesOE162+jfdvwmX3qXiwsYjpH6VTttbz1riAiItmiDubipqqUNhcyBZvVqrn55le56KIv6dr1EL16ZdK+/Qms1vPVycwBVjc2W6BWsYWTJzvW+75ZWWkcP55antyhbM9VB1u2DKFz52MkJZ31teIrxutl8OBtHD3aJeC2foZh5cyZZHbt6u+7Jhw50pmlSyfx7bejKCyUje7rI+xb8ADX8lalDTkqsmC2Z6xovGims5j5XE8vDlSqXWMAWaSxn54tErNoOgfj4cleZknfGAMuzYbbDta8LV5dFVsDFxozlH8tmGCYPPlhAGy2hxk1ytxv2O22sWTJFDZtGobHY6N3771MmbKE5577nwBX0A2qmX7yZPuAx91uB9nZaSgFN974Ku+/fwWHDnUDoHXrs1xxxYfEx9c8i8blcrJ/f08GDNjJggVXs39/T9xuGzabh0WLLuS6696ge/dD9Y45GkVEgi+ppapj2d+nFYiniMv4mGVMZCIrcOCiBCduHFzJe0h5gvByygn3DjPrpaOg0AIfpENWLPyxgbWoii3wURosSzETuc0Ad5VkroELzjQ2+uZht3u46KJFXHTRokrHhw/fwKZNwyv1wdvtbiZPXlbve7Rtm4MKMEnfZnPRoYM5nSghoYi5c/9LSYkDj8dGfHxRef2Y5OQznDoV+EXCavXQqtU5tm8f6EvuZrxls3EWLLian//871irri4TfiKii+Zp7gpYs706bTjH7czjIr7kIf7E3TxNZ47KHqth6N10325HFV6XXVZz1kl2A6o5l1rgnuHwUnfY2Rp2tzKTvNULaLNrxumFHx6BjoHfNIasiy/+krFjV2G3uwBNUlIu1177Jp061X9TjC5djpKcfAaLpWK3j4HN5mXo0M2Vzo2JcZGQUFSpONgVV7yP3V4KAWa5WSwGw4dvLt/PtSrDsHDsWHq9Y45GEZHg53EHH3E5RcTiCTgxsjILBnm0ZjXjeIz/5b/cQHE9XiAiVTjOoMlM9C/0BeYm1Ycb8F+6qAMcjzHLAZTxWsw9RqechAtyYMA5s1toRbvq69CEoqKiWHbuHIBSGrvdTWFhPCtXjsdd9e1JHSgFN930KhkZu7BYvChl0LXrEW677QXi4mp/5UtLO8FPfvIkY8aswukswWr14HCUEhdXyDXXvEVSUvXbR2itpJJkHUVEF42Blet4gwx2MJFlXMSXXMgiEigs738vU4KTj7icAhKDF7BoMr3yzb73qkneraBLA+pmrU42C3pV5TDM/vgtbc73va9tC2NyzPK+wc43kyc/XOuWfR99dDlnzlSeSXPkSBeWLZvE9OlL6n3P2NgSrrrqPQzjPbRWWK31G/RISCji4osXMXPmInJy2uJ22+jQ4WT5VM1hwzZz9Ghnv1a81eohPf1YveONRhHRgi+zkwE8y4+4kvdJJ5tXmYv2rXP1YKUEB0uZwm28EOxQRRO5Msu3V2mF3OLwmptlpNchwa9OhhtHw7TJcNVYyLObhcKq8gIbkysPrJZYYU1b8wUmFEye/HD5oGtVbreNfft6+U2T9HjsbNo0rFH3tViod3KvSClo1y6H1NTvKs3DHzBgO336ZGK3u8q34nM4Srn22gWNul80iYgWfCCfcQmjWIcNc5qWBS9uYriNF6T1HkE6lMI/N8E/e8OO1uA0YNZxuOtA7c9d3wZ+P+B8d0yO09ydqWoJYKXNGTklAXJKiQW+TYbBzbohZf2UJfkl3/yWzL6ZbBm8Be2xojcNgc8uper7DY8nNNOAxQJXX/0uWVlpHDjQnbi4YjIydhIbG2aDH0EUmv+zjTSYLQxnIzEVKkRaADsefsTT/Iaq+4OLcNarEP61uf611+d1r9zXDr4FTWUbC/jEe+DaI2Zt9qpbuNsNSAjBRZcaWHnXIyxLsuN2mlGrBdfAq3Ph7mfKz7NYvPTtuydIUdZNeno26emBd2wTNYuoLpoyvdmLJ8BrVwylDGJrECIKbZFS+72+38HRmgZh1fkPt8WXxAPcwAJMr33Xxxa3s5W5EKssuQPo+CK46VWsQzcCYLe7iI8v5MILF1V3GRHmIrIFv52B2P3aWlBEDGu5IAgRiaZQYoGl7eFQnLlB9eRTZpdMQ6UXw95AvXVVEnmpFT5Jgz9tg98OPN+4NxT8ahe0L/W7Qr0VW8067435fipam2xO+azK5iym+13PEP/Mj+jU6RiDBm3D6fT/WxGRISIT/B768TXTmM5i4ny7PXmxUEwczxJoNZ8IdSedcPdwc0FT2aYa87rDUxuhXQMrz95+wEzYlbppqunnKbCZe7C+txI2J5nJfejZxtd8PxgPf+0L+xLM247IhV/shuRG5twET+AFWjYNl095nsv6Pl9+rLbZNyJ8RWQXDcBVvMs/+QmnaUsRsXzMZYxmLTmEQAERUW//6A25DjO5g/nvGQf8u1fDrzk6F36zE9KLzIHUNqXmZtpV2QxzD1YwB1tH58KYM41P7uds8ONh5lx+r8Wc6rm+Dfx0WOPn1087GfiPWyuYVKUyQXUzb0T4i8gWPIALJw/yZx7kz8EORTSSxuxyqFoTxmuBVY2s6Dw+x/woa7ivbgu/zzDn0RsWc9VqosfcHq+pfdnRLGSmK3xfXgucdsCmJBhxtuHXbusy5+f/oX/lRP/7HdAqwKBwWZKX1nxkidgEL2oXTgOr1UXaVG9By64/Ngee3gjvpZsrWkfkwuzjjZsp41JmbZwkN7Sp0PVyOM5/Fg+YL2THYyFARet6GZcDH6wyu5SUNruUmqIAmwgfkuBFyFPAxNOwrJ3Zwi1jM8yB1qbWvRB+1sgNs8t8lArP9jTfIXgsMPIM/HoXxHmhfx583d5/5awCehY0zf2dxvmiaPk2mNfV3DvWbsDsbHOhmE2SfsSK2D54EVn+315IKzEHV22G+W96sbk/aaha2wae7gVFNnPMwO3rY/+jWeqc6SfN7p+KRREdXuibB/3ymzaWUgvcNdystHkyBrLizIJqDw+ofF5Nq2FF+JEWvAgLSW54aa1ZJfJIHHQrNPcrDeUWyhtd/GvGu62wPhly7WZ3zTMb4PkeZuEyuwGzTsCNh5u+ts3i9uagtLvCD6zUar7gHIg3p52KyCMJXoQNK+bslTEhWoe9qlPVbJNqN+Csw0zwyW54YI/50Zy2tg5cRE0BuxMlwUeqUG4ACRHWhp2t3P1SRmNOzWxJacVm909VSjfNQi0RmsIqwbfnO17iZnJJ4hTteIz7iUOaHiI03XAYYr2Vk7zTC3ceaPnZLJecgKoFGC0GtHbDsNzzx775pvaywyJ8hE0XTQzFrGU0qWTjwJyzdjdPMY7VjGMVwa/IHV7CaYpkuOpQCvPWw/yu5rz2dqVw3VEYHYQuprYueGwLPNrf7DrSCvrlmTN6GrO17NmzrdBakZR0TjbhCEEhk+BHsIGXGchP+CeLme73+DW8RTI55ckdIJZSBrKNCaxgBRNbMlwh6qRDadNNuWysjHx4ba25kMqhzdZ7mfq22k+eTOHtt68iN7cNAK1a5XH11e+QmnqiCSMWjVXvLhql1AtKqVVKqV/XcI5NKXVEKbXU91GnzU4HsoOPuJxRrPV7bCTrSQzQHWPFYLBUiBSiThSQ4jKTe1l3TH2Tu8tl46WXbubUqXZ4PHY8HjtnziTz8ss3UVJSzciyCIp6JXil1A8Aq9Z6HJCmlOpdzamDgTe01lN8H9vqeo9YivkNf/A7vot+ATfW9mBjH40oSCJElKlrUtfVjBPs2tUfr9dK5fShMAwL27cPCPwkERT17aKZAizwfb4YmADsDXDeGOAKpdR44DBwk9bab7G3UupO4E6ALr5jFjQD2OF3wfnM5RF+RwzF5XusurBxgg4s4sJ6fhtC1O6MHfYlQkqpubq1JXmBhanwcZpZ6mD6SbjyGMQ2sApZfVrpx46l89lns8jOTsPhcDFy5HqmTVuMzWbePD8/MeAuUG63nby8Vg0LUDSLGhO8UupZoG+FQ5OhfEPTPKi26bwOmKy1Pq6U+g9wCfBR1ZO01s8BzwGMVEoDeFFsYbDfBfNozThWMY/bGctqDCx8zkzuYB5Go4aJoosMrtZOA0/1hA/TzP1ePRYzwf9lW+V+6+b0aH9Y1e78QqnXYmFpCjyzsX6lBerb/XL6dFteeeXG8o2uXS4na9eOIi8vkauueh+ATp2OYbV6/PZ3dThcdO4sm2GHkhoTvNa6UvF0pdQ/gVjflwlU38WzVWtdNrt2N1BdV46fEmJ5hN8FfCyTvkxiOTEUY2DBhRPQTGQZPTjARoazLcCLgxD18UUHc4MPt/V8PfW9CWaJgb+1wHDPwThzN6aKhchcVsiONevxTGuC+juGAS6XA4fDhaXCX/HKleP8Wucej4Pdu/uTn/8ViYkFdO16hPT0bI4dS8fjMV8IbDY3KSmn6NkzhGtHRKH6dtFswOyWWQMMAapbf/eaUupPwHbgCuDR2i6sgfWM4Kf8g83UvMt7ie81JoWTLGEqXTgCaCxoljKZK/gAN466fk8iipyzwTftzcJbw3Ohf4CaL+908i8x4LXAliQ4azfLJjSnHa3NBUhVFdtgU5vaE3xtrfaFCy9i7doL0L46xb167eW6697AYoETJzqitX+7zWr1cOZMMomJBSgFN9wwn2+/vYBNm4ahtWLIkC2MHbum0ouFCL76JvgPgOVKqTRgFjBGKZUBXKe1rjir5hHgdcxB+4+01rVu+riREYxifb2CeZFb6M1eHBW255vKUn7Jn/lDNe8CRPTamAQP+eZzuZQ5P33caXhoV+W3ovn2wM+3aii0Nn+Cb+sK/NbY7oX2JYGfU9eumCVLJvPtt2OouG5k377evPbaDdx003xSU7M5caIDWld+hfN47CQn55R/bbMZjB+/mvHjV9fpviI46vV6q7XOwxxoXQNM1Vqf01rvrJLc0Vpv11oP1loP0lo/1HThnhdHIRfxVaXkbh4v5k6er+ZZIlp5lFk5scRqfhgW899VbWFZSuVzx54OXGIgzgOp1STYpjTqjLmzVNVWvBW4uMo084ozYkpLHWzaNJTly8dz5EjngLNgVq4cj/+iQMXBgz1wuy2MH78Ku73yfAibzcXAgdtJTJRV4+Gm3gudtNa5nJ9JEzSBNtUuE0ML/BWKsLK9FXgDjC+X2ODzjjClQrfHjUdgeYq5D6vLai7pt2v4+Z6Wqe1h0/CPzeZ+scdjzEQf7zVXnaa4ArfWs7NTeeWVGzEMhcdjw2bz0rXrYX74wzexVni1CjT7pUxeXmvats3l5ptfZuHCWWRlpeN0ljJq1DomT/6mGb5T0dxCZiVrfZ0jiV30ZxBbK/3RubDxAd8LWlyhKupnz9Tw7Vdt6Ca74MV18FEabGxjFuq6MqtlKy52LoaX1kFWjFnit0sRLP/mYQKlWa3hrbfmUFoaU37M7bZy+HBXNm4cxqhRG8qP2+0u3O7Ai5Fatz4HQFraCW677aUm/X5EcIT1kMgtvEQBiRRj/mIXEM9JOvBQ7WO6IsoMOAeWAF0WMV7/bg+A1h6YewSe2AL/mxm8crrpJXB44cMsr6GP/dSpdhQV+S8CdLsdbNxYecLCtGlL8H9J02Rk7Cif5y4iR9i24AE2MZw+ZHIrL9CP3axiHPOZSyEJwQ5NhBi7hod3wK8HmenNbTHnuI/NaZ5t/xqrPvPXta7p3Vnlx8aO/RbDsLBkyVQ8HhsWi8HQoZu4/PJPGxaoCGlhneABvqMjf6ZZxnFFhBlxFt5cA4tToMBubqjdPy/865C2b3+K2Nji8sVJZex2F0OHbvY7X2a/RI+wT/BCHI+B5e3Ob87dsYYx9tZuuCLb/7hLwbbW5jUGnTNb/MHQkFrsSsGcOW/z6qtz0VrhdttxOFykp2cxYsSG2i8gIpYk+AgX6YOr76Sbe5qW5eN53eGu/YGTeHXWtoFHBpy/hgJ+v8Ns4beUxm6y0alTFvfd9wTbtw+ksDCBLl0O0737IanRHuUkwYuwlRVjJndXlVWnz/Q0921VmBttJHpgdE7gXZRy7eZ0xNIq1/j1QHhztTnY2tyaagel2NjSSjNmhJAEL8LW8hQwArRQNfDXvrCrlTlzxoK5CvWxLdCnoPK5S9pXf/1v2sPl9XgnIESokQQvwlZ13eSGMuu5eKpMAr5rBHQpNKc/Tj9pHiuwmTNqqnIr87HmJHufiuYW1vPgRXSbcDrw3HZD+Sd3MPchPZwAj/WFBZ3MYyNzzemSVdk0jAzC3qlCNCVJ8CJsdS6GWw6Bw2vWjrEZ5udpRTU/r8QKL3czZ870zzMLjsVU6GuP8Zhz46t25zSVhmyTJ0RDSBeNCGvXHjVb8stSzJotk07D1tbwZG//kr9VnYqB9GKzmuSKdvBFR3Ng9uITMP50i4Tf7M6cSWLnzgF4vRb69t1Dx44ngx2SaEGS4CNYpE+RLNOpGK47cv7rDiXmph2ZiWYN9UC8CpJc5ucWzBeGSS2Q1Fuy5b5+/TA+/3wWhqHQ2sKyZRPJyNjJ7Nmf4nC00NZUIqgkwYuIY9Pw9y2wuh18nArr25jlgcs4vTD9O7NCY6TKz4/n889n4fGcL27v9VrYtm0wu3ZlMGPGV1xwwbogRihagiR4EZGsmF03E07Dovbwn15Q5OuyufgE3NPCO8u1dJ97ZmYfVKBtoVB4PHYWLbqQdu1y6NnzQIvGJVqWJHgR8S48CdNOmtvtJXgDz5qJNIGT+3lut4PVq8dIgo9wkuBFVLAAybV0OxvAaSfEeSGhiVawBmu2TN++mXz22SU1npOfn9hC0YhgkQQfgaJlcLUprU6Gv/c1N+PWypwD/+Dupkv0LS0+vojZsz/io48ux+u1UbVmptXqoXfvvcEJTrQYmQcvot6+ePj9AMhxmnVt3BZY1wYeGhjsyBpnyJDt/PSnTzJgwA6sVg9la3+tVg9xcUWMHSslgyOdtOBF1Hu7s1maoCKPFfYkwtFYc0FVuEpMLODqq9/l4MFurFo1hvz8VvTuvZcxY9YQHx/G35ioE0nwIuplxVaeRlnGpuGUs+EJPpRWq3bvfoju3Q8FOwzRwqSLRkS9oWfBHmBOvFsFby9WIZqCtOBF1LvyGHySCgUKvL4mT4wXZmdBUgMWfIZSy11EN0nwIuq1ccNzG+DlrrC2LbRyw1XHYNaJYEcmRONIgo8gMj2y4dqXwi8ygx2FEE0rohJ8Vw4RSzF76IuW4QXRwqRrRoSaiMiC3TnAJoaykwzWMYos0pnG18EOSwghgirsW/AWvCxlCulkYcUsMpJAIR/yPQawgyN0DXKEQggRHGHfgp/GYpI4W57cy9hxcxvzghSViDbSPSNCUdgn+DSyUQG2X3biohuHgxCREEKEhrBP8KsZixX/VSr5xLOIC4MQkRBChIawT/B76cNbzKGA+PJjxcRwhC4sYE4QI4tuhVb4tCPM7wKbkgjwHksI0dwaNMiqlOoAvKO1nljDOXbgfSAZmKe1frFhIdbuNl5kOZO4m6eIo4i3uIbHuZ9SYprrlqIGmQlw/1Bz39NSXxPComHQWbjzIPTPD2Z0QkSPeid4pVQb4BWo0GQO7MfAeq31w0qp95RSb2utm+VPW2PhJW7lJW5tjsuLetDAwwOgsMpvllfB5mS4rzU8vgUy8oISXpOTwVURyhrSReMFrgFq+xOdAizwfb4KGNmAe4kwcywWch3VP15qhWd7tFw8QkSzWlvwSqlngb4VDi3WWj+iVK3L4uOBLN/neUCHANe+E7jT/KpL7dGKiLAvIdgRCBEdak3wWuv/aeC1C4BY4ByQ4Pu66rWfA54DUGqkjMNFgE7F0MYFx2OrPyeltOXiaS7SNSPCQXPOotkATPB9PgQ41Iz3EsDUqZqpU4P7OqmAh3dAvAdsBn7TZ5xeuPFQEAITIgo1SakCpdQ0IENr/e8Kh18BPlNKTQQygG+b4l4i9PUpgLdWw5IU+LIj7E40jzsNuO0gTDsV3PiEiBYNTvBa6ykVPl8MLK7y+GGl1AzMVvxvtdYB9swRkSreC5edMD9cCgrs0NoN1gjoiJPuGREumrXYmNY6m/MzaUQLKeumCZX68A4Nya5gRyFE9An7layiesHujxdCBJckeCGEiFBhXw9eiJYife8i3EgLPsKFwtRJIURwSIKPEpLohYg+kuCFECJCSR+8ELWoS9+7YcC+fb3Zvn0ANpuHYcM207nzseYPTogaSIKPMqE2Rz4SaA3vvHMVe/f2wu12Agbbtg1iwoQVTJ68PNjhiSgmXTRCNNKBAz0qJHcAC263g2XLJnLuXKugxiaimyR4IWpQl+6Z3bv74nb7F8G3WDT79/dshqiEqBtJ8EI0ksPhQinD77hSGrtdajSI4JEEL0QjDR26BavVP8ED9OmT2cLRCHGeDLKKoCu2wGepsKqdWZTs+1kwIMh7ttZn1WpKymlmzfqMhQsvwWIxUMocyL722rdwOt3NFKEQtZMEL4Kq2Ap3DYfvYsz9WpWG5e3g3n1w2fFgR1d3I0ZsJiNjN/v398Bm89Cz5wHsdk+wwxJRThK8CKqPUs8ndwCtzM//3QumfwexgXs+QlJsbAkDB+4MdhhClJM+eBFUy1POJ/eKbBr2yAxDIRpFaR0a9UmUUqeAw3U8vR1wuhnDaW7hHH84xw4Sf7CFc/yhGntXrXVKoAdCJsHXh1JqvdZ6ZLDjaKhwjj+cYweJP9jCOf5wjF26aIQQIkJJghdCiAgVrgn+uWAH0EjhHH84xw4Sf7CFc/xhF3tY9sELIYSoXVi24JVSyUqpGUqpdsGORQgR+cI154RdgldKpQKfAqOBJUqpgNODQpVSqrVSaqFS6iul1PtKKf8yhCFOKdVBKSWFzltYuP7cw/13PpxzTtgleGAAcJ/W+k/AF8DwIMdTX9cDj2utZwAngIuDHE+9KKXaAK8A8cGOpb6UUi8opVYppX4d7FjqK5x/7oT57zxhnHPCLsFrrRdprdcopSZhvqKuDnZM9aG1fkpr/ZXvyxTgZDDjaQAvcA0Q5HJg9aOU+gFg1VqPA9KUUr2DHVM9heXPHcL/dz6cc07I16JRSj0L9K1waDHwB8xfdjfmL37IChS/1voRpdRYoI3Wek2QQquTGuIPVkgNNQVY4Pt8MTAB2Bu0aOpJa50HEIY/93Lh8jsfiDJ/8GGRcyoK+QSvtf6fah66Ryn1B+Ay4K0WDKleAsWvlEoG/gVc2fIR1U8NP/9wEw9k+T7PA3oFMZaoE06/84Foc7phWOScisKui0Yp9YBS6kbfl0nA2eBFU3++AaYFwK+01nWtvSMarwCI9X2eQBj+7oercP+dD+ecE46/5M8Bc5VSywAr8GWQ46mv24ARwENKqaVKqWuCHVCU2IDZLQMwBDgUvFCiTrj/zodtzpGFTiIqKKVaAcuBr4FZwBit9bngRiVE85IEL6KGb6rhDGCZ1vpEsOMRorlJghdCiAgVjn3wQggh6kASvBBCRChJ8EIIEaEkwQshRISSBC+EEBHq/wN9YpDvW9WqlgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import random\n",
    "import math\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib as mpl\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "\n",
    "# 定义类别转换函数\n",
    "def trtype(s):\n",
    "    types = {b'Iris-setosa': 0, b'Iris-versicolor': 1, b'Iris-virginica': 2}\n",
    "    return types[s]\n",
    "\n",
    "\n",
    "# 划分测试集和训练集\n",
    "def label_tr(input_label):  # 标签转换，将一维标签转换为三维\n",
    "    label_list = {0: [1, 0, 0], 1: [0, 1, 0], 2: [0, 0, 1]}\n",
    "    label_change = []\n",
    "    for i in range(len(input_label)):\n",
    "        label_change.append(label_list[int(input_label[i])])\n",
    "    return np.array(label_change)\n",
    "\n",
    "\n",
    "def inv_label_tr(input_label_change):  # 标签转换逆过程\n",
    "    y_real = []\n",
    "    for i in range(input_label_change.shape[0]):\n",
    "        for j in range(3):\n",
    "            if input_label_change[i][j] == 1:\n",
    "                y_lable = j\n",
    "        y_real.append(y_lable)\n",
    "    return np.array(y_real)\n",
    "\n",
    "\n",
    "def rand(a, b):  # 随机数函数 ---> 生成 a - b 之间的随机数\n",
    "    return (b - a) * random.random() + a   # random.random()用于生成随机的 0 - 1之间的浮点数\n",
    "\n",
    "\n",
    "def make_matrix(m, n, fill=0.0):  # 矩阵生成函数\n",
    "    mat = []\n",
    "    for i in range(m):\n",
    "        mat.append([fill] * n)\n",
    "    return mat    # 生成 m x n 的零矩阵\n",
    "\n",
    "\n",
    "def sigmoid(x):  # 激活函数\n",
    "    return 1.0 / (1.0 + math.exp(-x))\n",
    "\n",
    "\n",
    "def sigmoid_derivative(x):  # 激活函数求导\n",
    "    return x * (1 - x)\n",
    "\n",
    "\n",
    "class BPNeuralNetwork:  # BP神经网络类\n",
    "    def __init__(self):  # 初始化\n",
    "        self.input_n = 0   # 输入层\n",
    "        self.hidden_n = 0  # 隐含层\n",
    "        self.output_n = 0  # 输出层\n",
    "        self.input_cells = []  # 输入层节点数\n",
    "        self.hidden_cells = []  # 隐含层节点数\n",
    "        self.output_cells = []  # 输出层节点数\n",
    "        self.input_weights = []  # 输入层-->隐含层间的权重\n",
    "        self.output_weights = []  # 隐含层-->输出层间的权重\n",
    "        self.input_correction = []\n",
    "        self.output_correction = []\n",
    "\n",
    "        \n",
    "    def setup(self, n_in, n_hidden, n_out):   # 输入 2 5 3 \n",
    "        # 初始化输入、隐层、输出元数\n",
    "        self.input_n = n_in + 1  # n_in+1为输入层增加一个X0单元，代码中也加入了b值：a[1]=W.T*X+b\n",
    "        self.hidden_n = n_hidden\n",
    "        self.output_n = n_out\n",
    "        # 初始化神经元\n",
    "        self.input_cells = [1.0] * self.input_n   # 激活神经元\n",
    "        self.hidden_cells = [1.0] * self.hidden_n\n",
    "        self.output_cells = [1.0] * self.output_n\n",
    "        # 初始化权重矩阵\n",
    "        self.input_weights = make_matrix(self.input_n, self.hidden_n)  # 创建一个 input_n x hidden_n 的权重矩阵 3 X 5\n",
    "        self.output_weights = make_matrix(self.hidden_n, self.output_n)  # 5 X 3\n",
    "        # 初始化权重\n",
    "        for i in range(self.input_n):\n",
    "            for h in range(self.hidden_n):\n",
    "                self.input_weights[i][h] = rand(-0.2, 0.2)\n",
    "        for h in range(self.hidden_n):\n",
    "            for o in range(self.output_n):\n",
    "                self.output_weights[h][o] = rand(-2.0, 2.0)\n",
    "        # 初始化偏置\n",
    "        self.input_correction = make_matrix(self.input_n, self.hidden_n)\n",
    "        self.output_correction = make_matrix(self.hidden_n, self.output_n)\n",
    "\n",
    "        \n",
    "    def predict(self, inputs):   # inputs = x_train\n",
    "        #  激活输入层\n",
    "        for i in range(self.input_n - 1):  # i = 0 , 1 , 2\n",
    "            self.input_cells[i] = inputs[i]  # ----> input_cells = [x_test]\n",
    "        # 激活隐层\n",
    "        for j in range(self.hidden_n):\n",
    "            total = 0.0\n",
    "            for i in range(self.input_n):\n",
    "                total += self.input_cells[i] * self.input_weights[i][j]   # 神经元中是测试集的数据，乘以权重矩阵就是这一层的输入值\n",
    "            self.hidden_cells[j] = sigmoid(total)    # 隐层输出\n",
    "        # 激活输出层\n",
    "        for k in range(self.output_n):\n",
    "            total = 0.0\n",
    "            for j in range(self.hidden_n):\n",
    "                total += self.hidden_cells[j] * self.output_weights[j][k]  # 隐层输入 x 输入权重 = 输出层输入\n",
    "            self.output_cells[k] = sigmoid(total)    # 输出层输出\n",
    "        return self.output_cells[:]\n",
    "\n",
    "    \n",
    "    def back_propagate(self, x_train, label, lr, correct):  # cases = x_train[i]  label = y_train[i]\n",
    "        # 反向传播\n",
    "        self.predict(x_train)   # 得到的是输出层的输出\n",
    "        # 求输出误差\n",
    "        output_deltas = [0.0] * self.output_n  # [0, 0, 0]的数组\n",
    "        for o in range(self.output_n):\n",
    "            error = label[o] - self.output_cells[o]\n",
    "            output_deltas[o] = sigmoid_derivative(self.output_cells[o]) * error\n",
    "        # 求隐层误差\n",
    "        hidden_deltas = [0.0] * self.hidden_n\n",
    "        for h in range(self.hidden_n):\n",
    "            error = 0.0\n",
    "            for o in range(self.output_n):\n",
    "                error += output_deltas[o] * self.output_weights[h][o]\n",
    "            hidden_deltas[h] = sigmoid_derivative(self.hidden_cells[h]) * error\n",
    "        # 更新输出权重\n",
    "        for h in range(self.hidden_n):\n",
    "            for o in range(self.output_n):\n",
    "                change = output_deltas[o] * self.hidden_cells[h]\n",
    "                self.output_weights[h][o] += lr * change + correct * self.output_correction[h][o]\n",
    "                self.output_correction[h][o] = change\n",
    "        # 更新输入权重\n",
    "        for i in range(self.input_n):\n",
    "            for h in range(self.hidden_n):\n",
    "                change = hidden_deltas[h] * self.input_cells[i]\n",
    "                self.input_weights[i][h] += lr * change + correct * self.input_correction[i][h]\n",
    "                self.input_correction[i][h] = change\n",
    "        # 求全局误差\n",
    "        error = 0.0\n",
    "        for o in range(len(label)):\n",
    "            error += 0.5 * (label[o] - self.output_cells[o]) ** 2\n",
    "        return error\n",
    "\n",
    "    \n",
    "    def train(self, X_train, labels, epoch=10000, lr=0.05, correct=0.1):   # labels = y_train\n",
    "        # 训练神经网络\n",
    "        for j in range(epoch):\n",
    "            error = 0.0\n",
    "            for i in range(len(X_train)):\n",
    "                label = labels[i]\n",
    "                x_train = X_train[i]\n",
    "                error += self.back_propagate(x_train, label, lr, correct)\n",
    "        print(f\"error= {error}\")\n",
    "\n",
    "        \n",
    "    def fit(self, x_test):  # 离散预测函数用于输出数据\n",
    "        y_pre = []\n",
    "        for case in x_test:\n",
    "            y_pred = self.predict(case)   # return self.output_cells[:] ---> y_pred\n",
    "            for i in range(len(y_pred)):\n",
    "                if y_pred[i] == max(y_pred):\n",
    "                    y_pred[i] = 1\n",
    "                else:\n",
    "                    y_pred[i] = 0\n",
    "            y_pre.append(y_pred)\n",
    "        return inv_label_tr(np.array(y_pre))\n",
    "\n",
    "    \n",
    "    def fit2(self, x_test):  # 连续预测函数用于画图\n",
    "        y_pre = []\n",
    "        for case in x_test:\n",
    "            w = np.array([0, 1, 2])\n",
    "            y_pred = self.predict(case)\n",
    "            y_pre.append(np.array(y_pred).dot(w.T))\n",
    "        return np.array(y_pre)\n",
    "\n",
    "\n",
    "if __name__ == '__main__':  # 主函数\n",
    "    data = np.loadtxt('Iris.data', delimiter=',', converters={4: trtype})  # 读入数据，第五列转换为类别012\n",
    "    X, y = np.split(data, (4,), axis=1)  # 切分data和label\n",
    "    pca = PCA(n_components=2)\n",
    "    X = pca.fit_transform(X)  # 为方便绘图，对x进行PCA降维至二维\n",
    "    y = label_tr(y)\n",
    "    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1, train_size=0.8)  # 划分数据\n",
    "    \n",
    "    nn = BPNeuralNetwork()\n",
    "    nn.setup(2, 5, 3)  # 初始化网络\n",
    "    nn.train(X_train, y_train, 10000, 0.1, 0.1)  # 训练\n",
    "    \n",
    "    y_pre_1d = nn.fit(X_test)  # 测试\n",
    "    y_test_1d = inv_label_tr(y_test)\n",
    "    print(f\"测试分数为: {accuracy_score(y_pre_1d, y_test_1d)}\")  # 打印测试分数\n",
    "    \n",
    "    # 画图\n",
    "    x_min, x_max = X_train[:, 0].min(), X_train[:, 0].max()  # 第0列的范围\n",
    "    y_min, y_max = X_train[:, 1].min(), X_train[:, 1].max()  # 第1列的范围\n",
    "    xx, yy = np.meshgrid(np.linspace(x_min, x_max, 200), np.linspace(y_min, y_max, 200))  # 生成网格采样点\n",
    "    grid_test = np.stack((xx.flat, yy.flat), axis=1)  # 测试点  （xx.flat降维）\n",
    "    y_predict = nn.fit2(grid_test)\n",
    "    cm_pt = mpl.colors.ListedColormap(['r', 'g', 'b'])  # 样本点颜色（样本分为3个类，三个颜色）\n",
    "    cm_bg = mpl.colors.ListedColormap(['b', 'y', 'gray'])  # 背景颜色\n",
    "    mpl.rcParams['font.sans-serif'] = [u'SimHei']   # 设置字体为SimHei显示中文\n",
    "    mpl.rcParams['axes.unicode_minus'] = False  # 设置正常显示字符\n",
    "    plt.xlim(x_min, x_max)\n",
    "    plt.ylim(y_min, y_max)  # 设置坐标范围\n",
    "    plt.pcolormesh(xx, yy, y_predict.reshape(xx.shape), shading='auto', cmap=cm_bg)  # 绘制网格背景\n",
    "    plt.scatter(X_train[:, 0], X_train[:, 1], c=y_train, cmap=cm_pt, marker='o')  # 绘制样本点\n",
    "    plt.title(u'BPNN分类', fontsize=15)\n",
    "    plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
